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Abstract — The aim of this research is to design a car plate 
identification system by using the Speeded Up Robust Features 
(SURF) and the Support Vector Machine (SVM) methods. The 
system composes of two main stages including license plate 
localization and characters recognition. SURF can rapidly 
extract the interest points of an image because it uses the 
integral image technique to compute the determinant of the 
Hessian blob detector. SVM is a powerful machine learning 
technique to solve the classification problems which has been 
developed based on Structural Risk Minimization (SRM) 
principle. SVM has a good performance in character 
recognition. Therefore, it is very appropriate to be used in this 
system. By combining such two methods, this research has 
achieved 98.4% and 99.2% accuracy for plate detection and 
characters recognition, respectively. The plate verification and 
characters recognition have been performed in about 1.46 ms 
and 13.29 ms, respectively. 

Index Terms —Car plate identification, characters 
recognition, hessian matrix, integral image, SURF, support 
vector, SVM. 

I. INTRODUCTION 

With the development of vision technology, research on car 
license plate identification system has also been done 
intensively. Nevertheless, this field of research still leaves a 
very interesting challenge to be scrutinized. This is because 
the development of car license plate identification system 
requires the integration of various problem-solving 
techniques [1] to obtain the reliable results. The most 
fundamental problems often encounter are the accuracy and 
the processing speed. In addition, the non-uniform of car 
license plates and also the quality of lighting at the time of 
image capturing also become the challenges in developing 
this identification system. 

One of the solutions to meet the need of the above problems 
is, the authors has proposed a car license plate identification 
system by integrating SURF (Speeded-Up Robust Features) 
and SVM (Support Vector Machine) algorithms. SURF has 
been used because it can quickly extract the important points 
of an image by using the integral image technique on the 
Hessian matrix operation [2] [3]. SVM is a powerful computer 
learning system for solving classification problems that has 
been developed based on Structural Risk Minimization 
(SRM) principle of statistical learning theory [4] [5]. 
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Similar researches have been conducted as in [6] using 
Dual-Tree Complex Wavelet Transform (DTCWT) and 
Artificial Neural Network (ANN). This method has achieved 
94% accuracy for plate characters recognition. References [7] 
used the Hamming distance approach in characters 
recognition method that has resulted 95% accuracy. 
References [8] used the Global Direction Contributivity 
Density (G-DCD), Local Direction Contributivity Density 
(L-DCD), and Peripheral Direction Contributivity (PDC). 
This study has yielded 93.54% total accuracy. References [9] 
used Neural Network (NN) that has resulted a good accuracy 
at 97.78%, but this study was carried out for license plates of 
Czechoslovakia where they have a standard European 
countries license plate with a standard characters as well, 
which can be more easily detected. References [10] used the 
SVM by-means method that yielded 96.7% accuracy with 
57.4ms of time identification. References [11] used 
Probabilistic Neural Networks (PNNs), which achieved 
89.1% accuracy. References [12] used the template matching 
method, and it has reached 91.1% accuracy of the characters 
recognition. Reference [3] used SURF and Bag-of-Word, and 
Histogram Similarity has resulted 90.69%, 90.32% and 98% 
accuracy for images, videos, and webcam, respectively. 
Reference [13] used adaptive template matching which is 
implemented by using correlation method. This method has 
resulted 96% accuracy for the identification of the vehicle 
license plate in India. 



Figure 1. Flowchart of Indonesian car license plate 
identification system. 


In this paper, we present a new robust Indonesian license 
plate recognition system as shown in Fig.l. The system 
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composes of two main stages, namely plate localization and 
characters recognition. The plate localization stages is a 
stage to localize the region of license plate area, and the 
characters recognition stage is a stage to recognize the 
characters on the detected plate area. The processes of 
training data for both stages have been done in off-line mode, 
meaning that the data have been trained separately from the 
main system of car plate identification. The testing 
performance has been performed by using the car images 
were obtained with different backgrounds, illumination, 
license angles, distance from camera to cars, light conditions 
and different size and type of license plates. 

The rest of the paper is organized as follows: section 2 
presents the literatures study, section 3 explains the SVM 
training model, section 4 proposes the plate localization, 
section 5 proposes the characters segmentation and 
recognition, section 6 shows the result and discussion, and 
section 7 concludes the paper. 

II. Literatures Study 

A. Support Vector Machines (SVM) 

Basically, the objective of SVM is to create a hyperplane as 
a separator between two classes (i.e. +1 class and -1 class) 
with the largest margin [14] [15]. SVM was proposed for 
binary classification of pattern recognition techniques. 

Given a training set T = {x., y. }! =1 and 

U e R" , J. G {— l,l}) , where / is the number of data, n is 

the dimension of problem, and if the real function is 
determined by g(x), then the classification function can be 
obtained by [5]: 

f(x) = sign(g(x )) (1) 

It can divide the points in the R n space into two parts. If the 
function of g(x) is linear, then the hyperplane equation is [5]: 

g(x) = W.X + b (3) 

where w is the weight of the vector and b is the bias or 
threshold. Fig. 2 shows the linear hyperplane that separates 
the two classes. The parts that become the support vectors are 
marked with the larger circles. 



Figure 2. Linear hyperplane [14] 


SVM is not only expected to split the two classes perfectly, 
but it is also expected to classify the maximum intervals called 
maximum margin. Such separation is called optimal 
hyperplane division, so data can be classified into two 
optimization types as given below: [14] [16]. 

• 1 || ||2 

min - Ml (4) 

w,b Z 


S.t. y i (w.x i +b i )>V,i = \,...l 

By introducing the Lagrange multiplier Ot i /\ = 1,.and 

according to the principle of duality, the problem of (4) can be 
converted into dual problem [5]: 

m in . y ,y t a S *,(*, • *,) - a 


sj \ “ 

[a, * O.i m |,... J 

Given the solution of «;...«/ to the dual problem, solution to 
the original problem for w is: 

i 

w =Y. a >y>*> (6) 

i =1 

and b can be determined by: 

/ 

b = yj - x j) (7) 

i=l 

Non Linear SVM 

In practice, the data are not linearly separable, the 
hyperplane that maximizes margin can be done by minimizing 
misclassification error by introducing a slack variable 

£■ [14]. 

y,.(wjc.+£,.)> l-£ (8) 

If an error occurs, the corresponding must exceeds unity, so 
4^ is an upper bound on the number of training errors. So 
that the objective function (4) can be rewritten as: 

min IMf + c Zt i 6 } (9) 

where C is the parameter chosen by user to control the 
misclassification error. 

Multi Class SVM 

SVM has also been developed to solve multi-class cases. 
The most widely used method for multi-class SVM is to create 
N-SVM, where each SVM classifies one class from the other 
classes [16]. This method is called the one-versus-rest method 
[16]. In addition, the commonly used method is 
one-versus-one or called pairwise SVM [14], where it 
combines all possibilities of a two-class classification. For the 
N-class problem, the classifier training process should be 
done as much as A ^ ^ [16]. 

SVM Kernel 

In this work, we used two types of SVM kernel namely 
Linear kernel and Radial Basis Function (RBF) kernel. Linear 
Kernel is used in license plate localization process whereas 
RBF kernel is used in characters recognition process. The 
RBF kernels can be expressed in feature vectors in the input 
space defined as [17]: 

/ |k-^|| 2 \ 

K(x,x r ) = exp I - 2jZ j (10) 
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Where: \\x — x f \\ 2 is the Euclidean square distance between 
two feature vectors and a is the free parameter. By 
introducing y = — —z , RBF kernel is defined as [17]: 

K(x,x f ) = exp(y||x- x'|| 2 ) (11) 

LibSVM 

LibSVM is an open source library developed by 
Chih-Chung Chang and Chih-Jen Lin since the year 2000. It 
can be used to classify two-class and multi-class cases. Based 
on its literature [19], libSVM uses the one-versus-one SVM. 

B. Speeded Up Robust Features (SURF) 

SURF is a method used to extract local features from an 
image. It consists of a detector and a descriptor. The detector 
is used to locate the interest points (keypoints) and descriptor 
is used to extract the features in every detected keypoint. 
SURF uses the Hessian matrix approach to detect interest 
points by finding the maximum determinant value [2]. The 
Hessian matrix is used because it has good performance in 
terms of accuracy. The operation of Hessian matrix involves 
the integral image that allows for fast computation of box type 
convolution filters, so that the computational time decrease 
significantly [15]. The entry of an integral image I s (x) is 
generated by a square shape in image I at the location x = 
(x,y) T . This square area is formed by the origin and x as 
follows [2]: 

02 ) 

f=0 j-=0 



Figure 3.1ntegral Image [3] 


If given a point x = (x, y) in an image I, the Hessian H (x, a) 
matrix at point x with the scale a is defined as follows [2]: 
Axt(y,o) Lw(x,a) 


H (,v,0) = 


Lrv(jr,a) LyyiXiO) 


(13) 


where Lxx (x,<T) is the convolution of the Gaussian second 
d 2 f - , 

order derivative g (ff ) with the image I in point x, and 

similarly for Lxy (x,d) and Lyy (x,<T). 

In order to use the integral image in computation of 
convolution of the Gaussian second order derivative, SURF 
uses an approximation of boxes filter as shown in Fig. 4. 



Figure 4. Left to right: Gaussian filter in y-direction (Lyy) and 
xy-direction (Lxy) following by their approximation in 
y-direction (Dyy) and xy-direction (Dxy ) [2]. 


Dyy, and Dxy, respectively. Then, determinant of 
approximation of the Hessian matrix is written as follow [2]: 

detj )i. v | /->../> | '•/>-, (14) 

where w is the relative weight of the filter responses used to 
balance the effect of using box filter approximation. More 
details about this explanation is can be found in [2]. 


SURF Descriptor 

To determine the features, SURF uses Haar Wavelet 
responses in vertical direction dx and horizontal direction dy 
as shown in Fig. 5. The first step is to form a square area 
centered on the interest point. The size of this square is 2CB, 
where s is the scale at which the interest point is detected, then 
it is divided into 4x4 sub-region squares [2] as described in 
Fig. 6. 



Figure 5. Haar wavelet filter used to get orientation changes 
in pixel intensity around the interest point. The black weight is 
-1 and white weight is +1. 



Figure 6. Square area with 4x4 sub-region [2] 


For each of sub-region, convolution of pixels is performed 
with Haar wavelet response in direction of dx and dy. Then the 
orientation changes the intensity of the pixels. The obtained 
horizontal and vertical Haar Wavelet responses form a vector 
v = (E dx, E dy, E |dx|, E |dy|), which then produces a feature 
vector of 64 dimensions [2]. 



Figure 7. Illustration of SURF descriptor values for a 
sub-region [2] 


III. Training SVM Model 

Before plate localization and characters recognition can be 
performed, it is firstly required to conduct the training 
(learning) SVM data for both stages. In this work, the training 
data has been done in off-line mode, meaning that the data has 
been trained separately from the main system of car license 
plate identification. The objective of training SVM data is to 
classify the data based on given dataset. Then, the results of 
optimized training parameters, weight w and bias b can be 
used to classify the new unknown data. 


The approximation of convolutions of the Gaussian filter in 
x-direction, y-direction, and xy-direction are denoted to Dxx, 


Dataset Training 

The dataset for the process of license plate localization 
consist of two parts, including positive and negative dataset. 
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The positive dataset compose of license plate images, 
whereas the negative dataset compose of images which have 
totally no relation to the license plates, such as left/right car 
side, mirrors, lights indicators, tires, car doors, and so on. We 
have encountered difficulty to find the benchmarking dataset 
for Indonesian license plates, so we created our own dataset 
for both classes from the images captured by a camera. Fig. 
8.a and 8.b show the used positive and negative classes 
dataset in license plate localization process, respectively. 

For the characters recognition training dataset, we have 
used dataset from the tesseract [18] plus some of independent 
characters to fulfill the non-covered Indonesian style license 
plate characters. These independent characters are extracted 
from some images of Indonesian car license plates. The total 
of 36 classes have been labeled 0-9 and 10 - 35 
corresponding to 10 numbers from 0-9 and 26 alphanumeric 
characters from A - Z, respectively. 
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LibSVM, X t must be transformed into 1-D features vector by 
reshaping it, so that their elements become a features vector as 
follow: 


= lkl fu“ flM fn fu- ll£4 “““J&l (16) 

So, the features matrix F from all training images{Xj}-_ l9 

where / is the total of training data, can be written as follow: 
«-x n -, 
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Figure 8 Dataset training a) of class plate, and b) of class 
non-plate 

Feature Extraction 

The features extraction process has been performed by 
using the SURF method. For every image, SURF can result N 
strongest keypoints, and each keypoint consists of 64 features. 
Then, the total obtained features X t from an image is a 2-D 
matrix (size N x 64) as described below: 


Mg Idja - &Ij] —fijlj fcljj - 

Then, F can be directly trained by LibSVM along with their 
corresponding labels L as shown below: 


(18) 


In the case of binary classification of the license plate 
detection, y 1 through y t could be the number 0 or 1, where 0 
represents the non-plate and 1 represent the plate. Whereas, in 
case of multi-class classification of characters recognition, we 
labeled yj through y, with the numbers between 0 and 35, 
where 0-9 correspond to the number between 0-9, and 10 - 
35 correspond to 26 alphanumeric characters from A - Z. 

Training data for plate detection is done by using the linear 
SVM, whereas training data for characters recognition is done 
by using the RBF kernel. The used of RBF kernel in 
characters recognition is based on the results of the 
cross-validation as shown in table 1, where it appears that the 
RBF kernel provides maximum accuracy compared to 3 rd 
order Polynomial kernel and Sigmoid kernel. 

Table 1. Cross-validation performance for tree types of SVM 
Kernels used in characters recognition process. 


SVM Kernels 

C 

Gamma 

oo 

Cross-Validation 

Accuracy 

3 rd order 

Polynomial 

0.5 

2 

96.91 % 

Radial Based 

Function (RBF) 

8 

2 

99.47 % 

Sigmoid 

8 

1 

97.43 % 


In that way, the RBF kernel is better suited to our characters 
dataset, so we chose to use the RBF kernel in the case of 
characters recognition process. 

IV. Plate Localization 

The plate localization is a very important stage in the 
system of car license plates identification. This is due to the 
enormous variations between one image to another, such as 
backgrounds, illumination, license angles, distance from 
camera to cars, light conditions and different size and type of 
license plates. In addition, the quality of images also crucial 
important in determining the successful of the plate 
localization. 
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A. Pre-processing 

The initial step of the plate localization process is a 
pre-processing step, where it is the step to obtain the 
candidate plate areas from an image. The pre-processing steps 
consist of: 

1. Conversion original image into grayscale image. 

It is required to accelerate the detection process. In 
addition, the SURF features extractor works only with the 
grayscale images. Fig. 9.a shows the result of grayscale 
image. 

2. Tophat Morphology. 

It is used to find the difference between the grayscale image 
and the image after dilation and erosion processes. The 
purpose of this process is to obtain an image that has a higher 
brightness on the plate area. Fig. 9.b shows the result of the 
tophat morphology. 

3. Binarization 

This process is to change the image pixels to black and 
white. The higher contrast of image pixels will be changed to 
1 (white) and the darker part will be changed to 0 (black). So, 
that the candidate plate areas will be marked as white areas. 
Binarization process is done by using automatic thresholding 
OTSU method, which is a very effective method to be used 
because its capability to adapt to different lighting levels. Fig. 
9.c shows the image binarization result. 

4. Closing Morphology 

Closing morphology is applied with the aim is that to 
connect the existing characters in the plate area, so that the 
characters will be interconnected each other and form a 
detectable plate region. 




(a) 


(c)_(d) 


(e) 

Figure 9 Preprocessing results of a) grayscale, b) tophat 
morphology, c) binarization, d) closing morphology, and e) 
contours detection and size verification. 

5. Contours detection and Bounding box 




This step is to find the contours of the existing objects of 
the previous closing morphology image result. The objective 
of this step is to obtain the plate candidate areas. From any 
detected contours, it is applied a bounding box and verified 
its size, whether it meets the criteria of the license plate size 
or not. 

The criterions of the license plate has a minimum size of 
minwidth = 25 pixels, minheight = aspect x minwidth , and 
maximum size: maxwidth =125 pixels, maxheight = aspect x 
maxwidth , where aspect is the criteria of the ratio between 
height and width and its value is given by 2.5. The error is 5% 
of the aspect. So, the criteria of minimum size area is min = 
25 v aspect x 25 pixels and the maximum size area is max= 
125 x aspect x 125 pixels. The minimum and maximum 
respect ratio between height and width are rmin = aspect - 
aspect x error and rmax= aspects aspect x error , 
respectively. If area and r is the size area and the ratio of 
height and width of the evaluation plate candidate, 
respectively, then the pseudo code of criterion size 
verification is as follow: 

BEGIN 

IF((area<min OR area>max) OR (rcrmin OR r > rmax )) 

return FALSE 

ELSE 

return TRUE 

END 

If the contour does not meet the criteria, then it will be 
removed and only contours that meet the criteria will be 
retained as shown in Fig. 9.e. 

6. Crop candidate plate 

As a final step of the pre-processing stage is to crop the 
bounding box areas as the obtained plate candidates. Fig. 10 
shows the results of three obtained plate candidates. 



Figure 10. In the left side image shows that the candidate 
plates have been obtained, and in the right side images show 
the cropped plate candidates. 

B. Plate Verification 

Once the plate candidate has been obtained, then it should 
be classified to ensure whether it is true as a real license plate 
or not. This process involves SURF and SVM. The plate 
candidate is firstly extracted its features by using SURF 
method. Fig.ll.b shows the results of keypoints from each of 
the detected plate candidate in Fig. 1 l.a. These keypoints are 
then extracted their features by SURF descriptor and then 
classified by using bi-class linear SVM. 

V. Characters Segmentation And Recognition 

The characters recognition process is performed after the 
plate localization is completely accomplished and the plate 
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area has been already detected. Similar to the plate 
localization process, the characters recognition is also 
required the pre-processing stage to obtain the segmented 
characters. The pre-processing steps in characters recognition 
stage are as follow: 



Figure 12. Detected keypoints of characters B and 4 


1. Convert the detected plate area into gray-scale format. 

2. Apply the tophat morphology 

3. Image binarization by using automatic thresholding. 









a) Detected plate 
candidates 


b) SURF keypoints 


Figure 11. a) Detected plate candidates, and b) their 
corresponding detected keypoints. 


The process of characters segmentation consists of contour 
detection and bounding rectangle. Contour detection is 
applied to obtain the contour of the characters, then for each 
detected contour is applied a bounding rectangle. Each of 
bounding rectangle is verified its size. It must satisfy the 
aspect of comparison between height and width, where the 
minimum aspect is = 1.2 and the maximum aspect is 5.0. It 
also must meet the minimum and maximum height. Where the 
minimum height is 55 pixels and the maximum height is 95 
pixels. If these criterions are satisfied then the characters will 
be segmented and cropped to the further recognition process. 


Table 2 Result of characters segmentation for verified plates 


Klil 


Verified 

Plates 


Grayscale 


Binary 

images 


BL 433 JN 


Segmente 

d 

characters 


BL 433 J 


naan 


L H 3 HEH 




mm 


ft 69 ? jj 


01 Etfl AG 


l HHEH 


91 2/5 » 


697 jj 


Once the characters have been obtained, the SURF features 
extraction is applied on the same way of in the plate 
verification process. Then these characters are classified by 
using multi-class SVM with the RBF kernel. Fig. 12 shows 
the example of detected keypoints of characters B and 4. 


VI. Results And Discussion 


A. System Performance 

To evaluate the performance of the proposed system, The 
total of 250 images have been tested, resulted 100% accuracy 
of plate candidates segmentation. Then from these segmented 
plate candidates, they have been successfully verified by 
using SURF and SVM with the accuracy is 98.4% (246 out of 
a total 250 images), while four of them have failed to be 
recognized. Then from these 246 localized plates, they have 
been successfully recognized their characters with the 
accuracy is 99.2% (244 out of the total 246 localized plates). 
Some of the perfect car license plates identification are shown 
in Fig. 13, and details of result performance are shown in table 
3. 

The percentage of accuracy is calculated as follow: 

Number of Sllccvs^s 

Accuracy(^)= -——---——— - x 100 % 

total Number oil esled Images 


Table 3. Performance of car license plate identification 


Process Stages 

Number of 
Tested 
Images 

Number of Number of Accuracy of 
Successes Failures Successes 

Plate Candidate 





Segmentation 

(Preprocessing) 

250 

250 

0 

100.0% 

Plate 

Identification 

(SURF & SVM) 

250 

246 

4 

98.4% 

Character 

Identification 

(SURF & SVM) 

246 

244 

2 

99.2% 


Table 4 Execution times of each process 


Process 

Consumed Time Rate 

(miliseconds) 

Plate segmentation 

195 

Plate verification 

1.46 

Characters Segmentation 

52 

Cahracters Recognition 

13.29 

Total Time 

261.75 


The obtained results has indicated that the performance of 
Indonesian car identification plate system has a good 
accuracy that achieves 98.4% for plate detection and 99.2% 
for characters recognition as shown in table 3. The total 
execution time is 261.75 ms as shown in Table 4. The longest 
execution time occurs in the plate segmentation process that 
required 195 ms, while for plate verification involved SURF 
and SVM it just took only 1.46 ms. The characters 
segmentation process took about 52 ms, and for characters 
recognition process required 13.29 ms. Looking at the 
performance obtained, it can be concluded that as the name 
suggested the Speed-up Robust Features, SURF is a fast and 
robust features extraction, and SVM has good performance in 
identifying both the plates and the characters with a very good 
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accuracy. 



Figure 13. Results of perfect identification of the car license plates 


A. Problem Analysis 

We have identified the problem of misclassifications in the 
designed system as shown in Fig. 14. a and b. In Fig. 14.a, it is 
clear that the segmentation result of character 4 Q’ resembles 
character 4 0’. The character 4 Q’ has truncated that causes 
SVM recognized it as a character ‘O’. Similarly in Fig. 15 
where the character 4 B’ looks slightly defective. The 
segmented result of this character 4 B’ has a disconnected area 
as shown clearly in Fig. 15.b, so that SVM identified it as a 
number 4 8’. 

VII. Conclusion 

This study has developed an Indonesian car license plate 
identification system by integrating SURF (speeded-up robust 
features) and support vector machine (SVM) methods. 
By combining these two methods, this research has achieved 
98.4% and 99.2% accuracy for plate detection and characters 
recognition, respectively. The plate verification and 
characters recognition have been performed in about 1.46 ms 
and 13.29 ms, respectively. The results are very encouraging. 

For the perspective of this research, the authors need to 
improve the execution time of plate segmentation and also 
investigate a more robust pre-processing method, so that it 
can enhance the recognition performance, and this system can 
be used in a real time application system. 


a) 


BL 1779 nR 


b) 

Figure 14 a) Misclassification of Character 4 Q’, b) the result 
of characters segmentation. It is clearly that the result of 
character segmentation is not perfect. The character 4 Q’ has 
truncated at the bottom. 
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a) 


3L 178 AiP 


b) 

Figure 15. a) Misclassification of Q Character b) The result 
of characters segmentation. It is clearly can be seen that the 
result of character segmentation is not perfect. The Q 
character truncated at the bottom. 
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